Adaptive reinforcement learning for heterogeneous network selection

Next generation 5G mobile wireless networks will consist of multiple technologies for devices to access the network at the edge. One of the keys to 5G is therefore the ability for device to intelligently select its Radio Access Technology (RAT). Current fully distributed algorithms for RAT selection although guaranteeing convergence to equilibrium states, are often slow, require high exploration times and may converge to undesirable equilibria. In this dissertation, we propose three novel reinforcement learning (RL) frameworks to improve the efficiency of existing distributed RAT selection algorithms in a heterogeneous environment, where users may potentially apply a number of different RAT selection procedures. Although our research focuses on solutions for RAT selection in the current and future mobile wireless networks, the proposed solutions in this dissertation are general and suitable to apply for any large scale distributed multi-agent systems. In the first framework, called RL with Non-positive Regret, we propose a novel adaptive RL for multi-agent non-cooperative repeated games. The main contribution is to use both positive and negative regrets in RL to improve the convergence speed and fairness of the well-known regret-based RL procedure. Significant improvements in performance compared to other related algorithms in the literature are demonstrated. In the second framework, called RL with Network-Assisted Feedback (RLNF), our core contribution is to develop a network feedback model that uses network-assisted information to improve the performance of the distributed RL for RAT selection. RLNF guarantees no-regret payoff in the long-run for any user adopting it, regardless of what other users might do and so can work in an environment where not all users use the same learning strategy. This is an important implementation advantage as RLNF can be implemented within current mobile network standards. In the third framework, we propose a novel adaptive RL-based mechanism for RAT selection that can effectively handle user mobility. The key contribution is to leverage forgetting methods to rapidly react to the changes in the radio conditions when users move. We show that our solution improves the performance of wireless networks and converges much faster when users move compared to the non-adaptive solutions. Another objective of the research is to study the impact of various network models on the performance of different RAT selection approaches. We propose a unified benchmark to compare the performances of different algorithms under the same computational environment. The comparative studies reveal that among all the important network parameters that influence the performance of RAT selection algorithms, the number of base stations that a user can connect to has the most significant impact. This finding provides some guidelines for the proper design of RAT selection algorithms for future 5G. Our evaluation benchmark can serve as a reference for researchers, network developers, and engineers. Overall, the thesis provides different reinforcement learning frameworks to improve the efficiency of current fully distributed algorithms for heterogeneous RAT selection. We prove the convergence of the proposed reinforcement learning procedures using the differential inclusion (DI) technique. The theoretical analyses demonstrate that the use of DI not only provides an effective method to study the convergence properties of adaptive procedures in game-theoretic learning, but also yields a much more concise and extensible proof as compared to the classical approaches.Thesis (Ph.D.) -- University of Adelaide, School of Electrical and Electronic Engineering, 201

Nonlinear Control of Flexible Two-Dimensional Overhead Cranes

Considering gantry cable as an elastic string having a distributed mass, we constitute a dynamic model for coupled flexural overhead cranes by using the extended Hamilton principle. Two kinds of nonlinear controllers are proposed based on the Lyapunov stability and its improved version entitled barrier Lyapunov candidate to maintain payload motion in a certain defined range. With such a continuously distributed model, the finite difference method is utilized to numerically simulate the control system. The results show that the controllers work well and the crane system is stabilized

STUDY ON MULTI-OBJECTIVE OPTIMIZATION OF THE TURNING PROCESS OF EN 10503 STEEL BY COMBINATION OF TAGUCHI METHOD AND MOORA TECHNIQUE

In this study, the multi-objective optimization problem of turning process was successfully solved by a Taguchi combination method and MOORA techniques. In external turning process of EN 10503 steel, surface grinding process, the orthogonal Taguchi L9 matrix was selected to design the experimental matrix with four input parameters namely insert nose radius, cutting velocity, feed rate, and depth of cut. The parameters that were chosen as the evaluation criteria of the machining process were the surface roughness (Ra), the cutting force amplitudes in X, Y, Z directions, and the material removal rate (MRR). Using Taguchi method and MOORA technique, the optimized results of the cutting parameters were determined to obtain the minimum values of surface roughness and cutting force amplitudes in X, Y, Z directions, and maximum value of MRR. These optimal values of insert nose radius, cutting velocity, feed rate, and cutting depth were 1.2 mm, 76.82 m/min, 0.194 mm/rev, and 0.15 mm, respectively. Corresponding to these optimal values of the input parameters, the surface roughness, cutting force amplitudes in X, Y, Z directions, and material removal rate were 0.675 µm, 124.969 N, 40.545 N, 164.206 N, and 38.130 mm3/s, respectively. The proposed method in this study can be applied to improve the quality and effectiveness of turning processes by improving the surface quality, reducing the cutting force amplitudes, and increasing the material removal rate. Finally, the research direction was also proposed in this stud

Payload motion control for a varying length flexible gantry crane

Cranes play a very important role in transporting heavy loads in various industries. However, because of its natural swinging characteristics, the control of crane needs to be considered carefully. This paper presents a control approach to a flexible cable crane system in consideration of both rope length varying and system constraints. At first, from Hamilton\u27s extended principle the equations of motion that characterized coupled transverse-transverse motions with varying rope length of the gantry are obtained. The equations of motion consist of a system of partial differential equations. Then, a barrier Lyapunov function is used to derive the control located at the trolley end that can precisely position the gantry payload and minimize vibrations. The designed control is verified through extensive experimental studies

Catalytic Dye Oxidation over CeO2 Nanoparticles Supported on Regenerated Cellulose Membrane

A novel regenerated cellulose (RC) membrane containing cerium oxide (CeO2) nanoparticles is described in detail. In this work, CeO2 nanoparticles with high surface area and mesoporosity were prepared by a modified template-assisted precipitation method. Successful synthesis was achieved using cerium nitrate as a precursor, adjusting the final pH solution to around 11 by ammonium hydroxide and ethylene diamine, and annealing at 550 °C for 3 hours under a protective gas flow. This resulted in a surface area of 55.55 m².g–1 for the nanoparticles. The regenerated cellulose membrane containing CeO2 particles was synthesized by the novel and environmentally friendly method. The catalyst CeO2 and cellulose/CeO2 membrane were characterized by Fourier transform infrared spectroscopy (FTIR), X-ray diffraction (XRD), Electron paramagnetic resonance (EPR), and Brunauer-Emmett-Teller (BET) measurements. The g-value of 2.276 has confirmed the presence of the surface superoxide species of CeO2 nanoparticles in EPR. The photocatalytic activity of the catalyst and the membrane containing the catalyst was evaluated through the degradation of methylene blue under visible light irradiation by UV-VIS measurements. The cellulose/CeO2 membrane degraded 80% of the methylene blue solution in 120 minutes, showing a better photocatalytic activity than the CeO2 catalyst, which degraded approximately 62% in the same period. It has been proven that the RC membrane is not only a good transparent supporting material but also a good adsorption for high-performance of CeO2 catalyst. Copyright © 2022 by Authors, Published by BCREC Group. This is an open access article under the CC BY-SA License (https://creativecommons.org/licenses/by-sa/4.0).

Analysis of the effect of spray mode on coating porosity and hardness when spraying press screws by the high velocity oxy fuel method

Porosity and coating hardness are two very important properties of the coating. In order to achieve low coating porosity and high hardness, a suitable spray mode is desired. In the particular application for press screws with the complex surface, a suitable spray mode plays a significant role in the formation of the coating properties. This paper employs the Taguchi experimental design method combined with ANOVA analysis to evaluate the impact of the spray mode on the porosity and hardness of the coating while spraying the screw surface using the High Velocity Oxy Fuel (HVOF) method. The injection material used is WC HMSP1060-00 +60 % 4070, with its main components being Nickel and Carbide Wolfram. And the press screw material is 1045 steel. The impactful parameters of the spray mode investigated and tested are the flow rate of spray (F) with a range varying from 25 g/min to 35 g/min, spray distance (D) with a range of values varying from 0.25 m to 0.35 m, and an oxygen/propane ratio (R) from 4 to 6. The analysis shows that the spray mode significantly affects the coating properties, and a suitable set of spray parameters is found to achieve low coating porosity and high coating hardness. The spray mode with the lowest porosity is achieved at a spray rate (F) of 35 g/min, a spray distance (D) of 0.3 m, and an oxygen/propane ratio (R) of 6. The interactions between D and R, as well as between F and D, are statistically significant, influencing each other's effects on porosity. However, the interaction between F and R is relatively low, indicating that changes in one parameter have less impact on porosity when the other parameter is varied. Similarly, for the highest coating hardness, the optimal spray mode includes an F of 35 g/min, D of 0.25 m, and R of 6. There is a significant interaction between F and D, while the interaction between F and R is relatively low. Notably, there is no interaction between F and